Papers with sequence tagging
FITAnnotator: A Flexible and Intelligent Text Annotation System (2021.naacl-demos)
Copied to clipboard
| Challenge: | In this paper, we introduce FITAnnotator, a generic web-based tool for efficient text annotation. |
| Approach: | They propose a generic web-based tool for efficient text annotation. |
| Outcome: | The proposed tool is based on a fully modular architecture and provides three kinds of interfaces to annotate instances, evaluate annotation quality and manage the annotation task for annotators, reviewers and managers. |
FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks (2022.findings-naacl)
Copied to clipboard
Bill Yuchen Lin, Chaoyang He, Zihang Ze, Hulin Wang, Yufen Hua, Christophe Dupuy, Rahul Gupta, Mahdi Soltanolkotabi, Xiang Ren, Salman Avestimehr
| Challenge: | Increasing concerns and regulations about data privacy necessitate the study of privacy-preserving, decentralized learning methods for natural language processing tasks. |
| Approach: | They propose a framework for evaluating federated learning methods on four different tasks . they propose federation between Transformer-based language models and FL methods . |
| Outcome: | The proposed framework compares FL methods on four different tasks under non-IID partitioning strategies. |
PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains (2022.tacl-1)
Copied to clipboard
| Challenge: | Domain Adaptation (DA) algorithms suffer degradation when applied to out-of-distribution examples. |
| Approach: | They propose an example-based autoregressive Prompt learning algorithm for on-the-fly Any-Domain Adaptation . the algorithm is trained to generate a unique prompt that maps the test example to a semantic space . |
| Outcome: | The proposed model outperforms baselines in 14 multi-source adaptation scenarios. |
LAraBench: Benchmarking Arabic AI with Large Language Models (2024.eacl-long)
Copied to clipboard
Ahmed Abdelali, Hamdy Mubarak, Shammur Chowdhury, Maram Hasanain, Basel Mousi, Sabri Boughorbel, Samir Abdaljalil, Yassine El Kheir, Daniel Izham, Fahim Dalvi, Majd Hawasly, Nizi Nazar, Youssef Elshahawy, Ahmed Ali, Nadir Durrani, Natasa Milic-Frayling, Firoj Alam
| Challenge: | Recent advances in Large Language Models (LLMs) have significantly influenced the landscape of language and speech research. |
| Approach: | They used GPT-3.5-turbo, GPT-4, BLOOMZ, Jais-13b-chat, Whisper, and USM to tackle 33 distinct tasks across 61 datasets. |
| Outcome: | The proposed model outperforms SOTA models in zero-shot learning, with a few exceptions. |
Efficient Encoders for Streaming Sequence Tagging (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing bidirectional encoders require a restart when a new token is received. |
| Approach: | They propose a Hybrid Encoder with Adaptive Restart that enables asynchronous encoding of a new token in an incremental streaming input. |
| Outcome: | The proposed encoder offers FLOP savings in streaming settings up to 71.1% and outperforms bidirectional encoders for streaming predictions by up to +0% streaming exact match. |
ALLECS: A Lightweight Language Error Correction System (2023.eacl-demo)
Copied to clipboard
| Challenge: | We present a lightweight web application to serve grammatical error correction systems . ALLECS is designed to be accessible to as many users as possible . |
| Approach: | They propose a web application that can serve grammatical error correction systems . they propose to provide three state-of-the-art base GEC systems and two combine methods . |
| Outcome: | The proposed system can be easily used by the general public and is available for free on nus.edu.sg. |
ALToolbox: A Set of Tools for Active Learning Annotation of Natural Language Texts (2022.emnlp-demos)
Copied to clipboard
Akim Tsvigun, Leonid Sanochkin, Daniil Larionov, Gleb Kuzmin, Artem Vazhentsev, Ivan Lazichny, Nikita Khromov, Danil Kireev, Aleksandr Rubashevskii, Olga Shahmatova, Dmitry V. Dylov, Igor Galitskiy, Artem Shelmanov
| Challenge: | Currently, the framework supports text classification, sequence tagging, and seq2seq tasks. |
| Approach: | They propose an open-source framework for active learning annotation in natural language processing that provides an easy-to-deploy GUI annotation tool directly in the Jupyter IDE. |
| Outcome: | The proposed framework reduces computational overhead and duration of AL iterations and increases annotated data reusability. |
Model Compression for Domain Adaptation through Causal Effect Estimation (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for compressing language representation models are not interpretable and do not consider the differences in the predictive power of various model components or the generalizability of the compressed models. |
| Approach: | They propose a model compression scheme that estimates the average treatment effect of a single layer on the model's predictions. |
| Outcome: | The proposed model compression scheme outperforms strong baselines on dozens of domain pairs across three text classification and sequence tagging tasks. |
Is ChatGPT a General-Purpose Natural Language Processing Task Solver? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in scale have enabled large language models to perform NLP tasks zero-shot . however, it is not known whether ChatGPT can serve as a generalist model that can perform many NLP jobs zero- shot. |
| Approach: | They empirically evaluate ChatGPT's zero-shot learning ability on 20 popular NLP datasets . they find it performs well on many tasks favoring reasoning abilities . |
| Outcome: | The proposed model can perform many NLP tasks zero-shot without adaptation on downstream data. |
A Bayesian Approach for Sequence Tagging with Crowds (D19-1)
Copied to clipboard
| Challenge: | Existing methods for sequence tagging are data hungry and annotators are unreliable . current methods do not account for common types of span annotation error . |
| Approach: | They propose a Bayesian method for aggregating sequence tags that models sequential dependencies between annotations and ground-truth labels. |
| Outcome: | The proposed method outperforms existing methods on crowdsourced data and reduces crowdsourcing costs through active learning. |
Semi-Supervised Tri-Training for Explicit Discourse Argument Expansion (2020.lrec-1)
Copied to clipboard
| Challenge: | a novel application of semi-supervision for shallow discourse parsing is described . we focus on explicit discourse arguments, but we leave the sense selection aside . |
| Approach: | They propose a semi-supervised approach for shallow discourse parsing using sequence tagging. |
| Outcome: | The proposed approach improves performance by 2-10% in the first setting and by comparing the results with training relations. |
Distillation of encoder-decoder transformers for sequence labelling (2023.findings-eacl)
Copied to clipboard
| Challenge: | despite the strong trend in NLP to explore the use of large language models, there is still limited work evaluating prompting and decoding mechanisms for SL tasks. |
| Approach: | They propose a hallucination-free framework for sequence tagging that is especially suited for distillation. |
| Outcome: | The proposed framework performs well across multiple sequence labelling datasets and in a few-shot learning scenario. |
TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance (2021.acl-long)
Copied to clipboard
Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, Tat-Seng Chua
| Challenge: | Existing QA systems focus on unstructured text, structured knowledge base, or semi-structured tables. |
| Approach: | They propose a large-scale question answering model based on financial reports . numerical reasoning is usually required to infer the answer . |
| Outcome: | The proposed model achieves 58.0% inF1, an 11.1% increase over the baseline model, but still lags behind the best human model. |
Comparing Machine Learning and Deep Learning Approaches on NLP Tasks for the Italian Language (2020.lrec-1)
Copied to clipboard
| Challenge: | Using available datasets, we compare deep learning and traditional machine learning methods for various NLP tasks in Italian. |
| Approach: | They compare deep learning and traditional machine learning methods for various NLP tasks in Italian. |
| Outcome: | The proposed methods outperform traditional methods in sequence tagging tasks and classification tasks in Italian. |
Explicit Alignment Objectives for Multilingual Bidirectional Encoders (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained cross-lingual encoders have proven impressively effective at enabling transfer-learning of NLP systems from high-resource languages to low-resourced languages. |
| Approach: | They propose a method to align multilingual encoders using two explicit alignment objectives that align the multilingual representations at different granularities. |
| Outcome: | The proposed method achieves gains of up to 1.1 average F1 score on sequence tagging and 27.3 average accuracy on retrieval over the XLM-R-large model. |
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks (2022.emnlp-main)
Copied to clipboard
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A, Sumanta Patro, Tanay Dixit, Xudong Shen
| Challenge: | a benchmark of 1,616 diverse NLP tasks and their expert-written instructions is used to test generalization of models to unseen tasks . a recent study shows that instruction-following models outperform instruction-based models by over 9% . |
| Approach: | They build a benchmark of 1,616 diverse NLP tasks and their expert-written instructions. |
| Outcome: | The proposed model outperforms existing instruction-following models by over 9% on the benchmark despite being smaller. |
SciBERT: A Pretrained Language Model for Scientific Text (D19-1)
Copied to clipboard
| Challenge: | SciBERT is a pretrained language model based on BERT to improve performance on scientific NLP tasks. |
| Approach: | They propose a pretrained language model based on BERT to improve NLP performance . they evaluate on sequence tagging, sentence classification and dependency parsing . |
| Outcome: | The proposed model improves on sequence tagging, sentence classification and dependency parsing tasks with datasets from a variety of scientific domains. |
A Pointer Network Architecture for Joint Morphological Segmentation and Tagging (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Morphological Disambiguation (MD) is a task of decomposing tokens into morphemes . a simple pipeline is used to segment and tagging raw tokens . |
| Approach: | They propose a new pointer network model that combines symbolic knowledge of morphemes with the learning capacity of neural end-to-end modeling. |
| Outcome: | The proposed model outperforms all previous reported results on Hebrew and Turkish . it uses morphological knowledge and the learning capacity of neural end-to-end modeling . |
Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on simple approaches to model the output label space . graphical models are often limited to (heuristic) greedy search and its variants . |
| Approach: | They propose an approach for efficiently training and decoding hybrids of graphical and graphical models based on Gibbs sampling. |
| Outcome: | The proposed approach improves on Dutch and Dutch with graphical models . the proposed model improves over a strong baseline on three languages . |
SuperGLEBer: German Language Understanding Evaluation Benchmark (2024.naacl-long)
Copied to clipboard
| Challenge: | a new set of German-pretrained models are being released, but no established, diverse and systematic evaluation suite is available for them. |
| Approach: | They assemble a Natural Language Understanding benchmark suite for the German language and evaluate 10 existing German-pretrained models. |
| Outcome: | The proposed benchmark suite evaluates 10 German-pretrained models on 29 tasks . the results show that encoder models are good choices for most tasks, but not all . |
Scientific Keyphrase Identification and Classification by Pre-Trained Language Models Intermediate Task Transfer Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Scientific keyphrase identification and classification is challenging due to lack of labeled data for training deep neural networks. |
| Approach: | They propose to use pre-trained language models BERT and SciBERT to train scientific keyphrase identification and classification using intermediate task transfer learning. |
| Outcome: | The proposed model achieves competitive performance in scientific keyphrase identification and classification compared to both previous studies and strong baselines. |
Identification of Fine-Grained Location Mentions in Crisis Tweets (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies have focused on identifying informative tweets by individuals affected by a crisis, without considering their specific types. |
| Approach: | They assemble two tweet crisis datasets and manually annotate them with specific location types to facilitate progress on the fine-grained location identification task. |
| Outcome: | The proposed model performs well in both in-domain and cross-domain settings. |
Transforming Sequence Tagging Into A Seq2Seq Task (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained, large, generative language models have had great success in a wide range of sequence tagging and structured prediction tasks. |
| Approach: | They propose to use a new format for casting input text sentences and their output labels into the input and target of a Seq2Seq model and introduce it to test their hypothesis. |
| Outcome: | The proposed format shows to be both simpler and more effective and devoid of hallucination. |